Repository navigation
fix(sqlite): apply each WAL commit in one Ladybug transaction - #142
Merged
Merged
Conversation
One SQLite WAL commit now maps to one Ladybug transaction: the batch's statements plus its cursor advance commit atomically. A mid-tick failure (e.g. a binder error on one edge table) leaves no partial rows behind, and the next tick replays whole batches idempotently. - applyBatchTx wraps BEGIN .. batch .. cursor .. COMMIT with ROLLBACK on any statement failure; stats only count committed work. - Cursor advances per batch instead of once at tick end, so crash recovery resumes from the last committed SQLite commit. - DDL stays outside the transaction (replay-safe: IF NOT EXISTS / one-shot shadow healing).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One SQLite WAL commit now maps to one Ladybug transaction: the batch's statements plus its cursor advance commit atomically.
Motivation.
SyncEngine::syncran every statement through the inner context in autocommit, so a tick with N statements was N transactions. A mid-tick failure left committed prefixes behind — observed live: node upserts for a staged batch committed, then the edge batch failed withBinder exception: Cannot find property _sqlite_rowidon a pre-existing rel table, leaving a half-replicated batch (duplicate memory + entity rows after the fallback applier converged it).Change (
sqlite/src/function/sync_engine.cpp):applyBatchTxwrapper:BEGIN… batch statements … cursor upsert …COMMIT, withROLLBACK(and stats-restore) on any statement failure.CREATE … IF NOT EXISTS, shadow healing) stays outside the transaction: replay-safe by construction.Validation.
ladybug-memory-rs,sqlite_first): against the pre-fix build the failed tick leaves(mem, ent, men) = (1, 1, 0); with this build it leaves(0, 0, 0), and the nodes-only batch commits cleanly (via=extension, cursor advanced).sync_api_testcould not be built in this environment (no gtest available to the standalone extension build); the cdc unit tests cover untouched code. Recommend CI run the full extension suite.Follow-up (not in this PR): edge
_sqlite_rowidslot on pre-existing rel tables — tracked separately.